For convenience and clear understanding of the presented invention, the terms used in this paper have the following meanings.
Search engine means a computer program designed for the search of information on the Internet. The search is conducted according to a query of the user. The results of the search are provided to the end-used in a form of ranking according to some relevancy index. The examples of search engines are Microsoft Bing search engine, Google search engine, Yahoo search engine or Yandex search engine.
Recommendation system means a computer program that selects from some presented set of alternatives (options) those ones which can be interesting according to some criteria for a particular user of the system. It should be noted that in most cases the results of recommendation systems are either a set of recommended options or ranking of all of these options or a part of them. Different methods of information processing and transformation which are used in recommendation systems can be applied to other related fields, for instance, to the problem of efficiency evaluation of enterprises, etc.
Superposition (in the present context, in contrast to the known term of superposition in physics) means a sequential elimination of options from the initial set according to some rules which can be different in different stages. The examples of such rules are provided in Appendix 1. In the first stage the elimination of options is made from the initial set of alternatives while in further stages it is made from the remained set of effective options.
Efficient (“good”) elements (options, items) mean elements, which are the best, most preferred or most useful according to some given criteria when it is necessary to make a ranking of options for satisfying the needs of the users (individuals, experts, agents).
Inefficient (“bad”) elements (options, items) mean elements, which obviously never (under no circumstances) can be used for problem solving since there are more preferred alternatives.
The efficiency (utility) value according to which a set of selection and ranking rules is defined is given by experts.
Most search engines have some tools for data storage and processing that contain efficiency evaluations (relevance) of large representative sets of queries and search results to these queries. In such tools the query and the results of the query (a set of options) are characterized by a set of criteria as well as by relevance evaluations assessed by the experts (assessors).
There are various formal criteria for evaluating the relevance of the search element to the query (for instance, the term frequency or TF-IDF which take into account the frequency and the importance of each term of the query). It should be noted that these formal criteria are rather algorithms which are used by search engines than independent criteria which evaluate the results of the search. Evaluations calculated by such formal criteria can still be very different comparing to evaluations given by experts.
There are three main approaches for the selection and ranking of options.
There is an approach for selection and ranking of options when a numerical or ordinal score is assigned to each option according to some criteria. The score of each option represents its relative importance. The most frequently used method is a regression one.
Another example is McRank algorithm which consists in calculating so-called “expected relevance” for each pair “query-option” which is a function of the probability that the option belongs to some class of relevance. All options are ranked according to the calculated values of the “expected relevance” [Ping L., Burges C J C, Wu Q. McRank: Learning to Rank Using Multiple Classification and Gradient Boosting. NIPS. Curran Associates, 2007].
There is an approach for the selection and ranking of options which consists in the pairwise comparison of options. The selection of efficient options is made according to the binary classifier. The examples of this approach are support vector machine models, which construct a hyperplane or a set of hyperplanes in a high- or infinite-dimensional space with the largest distance to the nearest option of any relevance class [S. Cortes, V. N. Vapnik; “Support-Vector Networks”, Machine Learning, 20, 1995]. Other examples of this approach are:    RankNet [Christopher J. C. Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, Gregory N. Hullender: Learning to rank using gradient descent. ICML 2005: 89-96] which use a neural network and a simple probabilistic cost function for the ranking of options.    RankBoost [Y. Freund, R. Iyer, R E Schapire, and Y. Singer. An efficient boosting algorithm for combining preferences. J. Mach. Learn. Res., 4: 933-969, 2003] which consists in the sequential composition of machine learning algorithms for the classification of pairs of options.    FRank [M. Tsai, T.-Y. Liu, et ah FRank: A Ranking Method with Fidelity Loss, SIGIR 2007] which is a modification of the RankNet method that instead of simple probabilistic cost function use a fidelity loss function, etc.
There is an approach for the selection and ranking of options which consists in a listwise comparison of options. The examples of this approach are
1. A method for constructing trees and minimization of the loss function ListNet, which introduces a probability space on the set of permutations. The entropy function is used as the loss function in this method. [Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. Learning to Rank: From Pairwise Approach to Listwise Approach, 2007],
2. The method of the listwise comparison of options RankCosine, which uses a loss function based on the cosine similarity between the ranking list of options and the corresponding ground truth [T. Qin, X.-D. Zhang, M.-F. Tsai, D.-S. Wang, T.-Y. Liu, H. Li: Query-level loss functions for information retrieval. Inf. Process. Manage. 44 (2): 838-855, 2008].
3. The ranking method AdaRank that uses the machine learning algorithm AdaBoost. AdaBoost is an algorithm that linearly combines classifiers for making ranking predictions. [Xu, J., Li, H.: AdaRank: a boosting algorithm for information retrieval. SIGIR 2007],
4. The ranking method SoftRank which performs a direct optimization of non-smooth ranking metrics [Michael Taylor, John Guiver, Stephen Robertson, and Tom Minka. SoftRank: Optimising Non-Smooth Rank Metrics, 2008], etc.
All these methods produce some good results in particular areas. The disadvantages of these methods for the selection and ranking of options are:                The use of sophisticated procedures on large datasets which leads to the significant increase in the computational complexity;        Low accuracy of selection and ranking of options in case of large number of criteria and/or options.        
Decision trees are usually used on large datasets. The decision tree method consists in the construction of a sequence of threshold procedures on different criteria. Decision trees have some disadvantages. First, the results of decision trees have a low reliability, since the use of threshold procedures is not always justified (effective). Second, decision trees do not allow the use of a group of criteria at the same stage. Finally, it is often difficult to perform the selection or ranking of all options with the use of only one or multiple criteria. Thus, it is necessary to construct a large number of different decision trees and aggregate their results.
Some methods are known from the Russian Federation patents No 2435212 (“Data collection of user behavior when searching the web to improve the search relevance”—[“Sbor dannih o polzovatelskov povedenii pri veb-poiske dlya povysheniya relevantnosti poiska”]), No 2443015 (“Ranking functions using Bayesian classifier of queries with incremental update”—[“Funktsii ranzhirovaniya, ispolzuyushchie modifitsirovanniy bayesovskiy klassifikator zaprosov s incrementnym obnovleniem”]), No 2367997 (“Improved systems and methods for the ranking of documents on the basis of structurally linked information”—[“Usovershenstvovannie sistemi i sposoby ranzhirovaniya dokumentov na osnovanii strukturno vzaimosvyazannoy informatsii”]) which consists in gathering some additional information (Bayesian classifier, user behavior, structural relationships of documents) for the selection and ranking of options. The main disadvantage of these methods is that all of them make existing methods of selection and ranking of options more complicated as some additional criteria have to be used.
The method most related to our invention relies on the calculation of temporary weights for the search results. On the first stage the user event corresponding to the search result is identified. The user event has the start time, the end time and the duration. On the next stage the current time is determined and the temporary weights for the search results are calculated on the basis of the temporal proximity of the current time to the user event. It is assumed in the method that the temporal weight varies with time: it increases exponentially as the current time approaches to the start time of the event, it is constant during the event and it decreases after the end time of the event. The method is designed for the search of information on the Internet using a temporary weights for the search results (Russian Federation patent No 2435213, IPC G06F 17/30, 27 Nov. 2011).
The disadvantage of this method as well as the disadvantage of other similar search techniques is that it tends to apply ‘approximate’ algorithms for the selection and ranking of options, i.e. algorithms with linear or close to linear computational complexity O(n), where n is the number of options. The decrease of complexity is usually achieved by means of simplification (or more precisely, by approximation) of the developed algorithms. However, such simplification reduces the accuracy of the results.